Auto-associative Learning of On-line Handwriting using Recurrent Neural Networks

Traditionally the parametric grammar based approach to the modeling and recognition of temporal sequences using hidden Markov models HMMs involves a very crucial step which requires human experts to determine a priori the appropriate model architecture to work on This includes among other things determining the number of states in the proba bilistic grammar and the probabilistic transitions between states As a long term e ort we attempt to develop a more domain independent principled approach to the modeling of grammatical structures in temporal sequences without knowing in advance the topology of the underlying grammars This is achieved through an unsupervised learning process In particular a discrete time recurrent neural network model called ASCOC we proposed before is trained to learn separately the dynamics of each individual embedded subgrammar or subpattern class These subgrammar network models are trained in an auto associative or self supervised manner which is similar in spirit to the principal component analysis PCA learning paradigm for feedforward neural networks except that our focus here is on recurrent neural networks that model dynamical behaviors In the pilot study presented in this paper some issues of this new approach to temporal sequence processing are investi gated in the domain of on line handwriting modeling and recognition Some possible future research directions are also discussed